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ABSTRACT 

This study, part of the Colorado Science f ( 
Meta-Analysis Project, was designed to assess the effectiveness of 
teacher education programs' on preservice and inservice teachers 
having as measured outcomes one or more variable's associated with 
inquiry teaching. Inquiry teaching within the context of the study 
addresses teacher behaviors^ facilitating student acquisition of 
concepts, processes, and skills. through active involvement with 
general inquiry strategies." Specific outcome criteria which' appeared 
in the research reviewed for the analysis and judged as falling 
Within the confines of this definition include: knowledge of science 
processes, inquiry instructional strategy, indirect verbal behavior, 
* accepting interpersonal behaviors, increased wait-time questioning 
behavior, higher cognitive questioning behavior, and discovery 
instructional strategy. Sixty-eight studies, involving at least one 
of these outcomes associated with inquiry strategy, were selected for 
analysis. Relevant variables (114) were identified and coded ^ 
according to: study form/design characteristics, teacher/teacher 
trainee characteristics, student characteristics, treatment 
characteristics, outcome characteristics, and effect size calculation 
characteristics. Because descriptive reporting of teacher and/or 
Student characteristics which might affect outcome measures was 
sporadic and occurred with frequencies too low to support analysis, 
the analysis was divided into categories focusing on variables 
associated with: methodological aspects of studies, study treatments, 
and the variety of teacher education outcomes sought. (JN) 
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INTRODUCTION 



This meta-analysis study was a part of the Colorado Science Meta-Analysis 
Project (Anderson, 1982) designed to provide a synthesis of research 
related to science education on a number of important questions related to 
the teaching and learning of science and the preparation of teachers of 
science. 

One of the important , science education concerns has been the effects of 
different preservice and inseryice science teacher education approaches on 
teachers and, ultimately, the effects of teachers on their students. 
During the 1960 f s and 1970' s several of the curriculum projects developed 
materials designed to b6 taught in an inquiry style that was investigative 
and student centered; inquiry skills were also desired outcomes of 
instruction. This study was designed to assess the effectiveness of 
teacher education programs on preservice and inservice teachers having as 
measured outcomes one or more variables associated with inquiry teaching. 

Inquiry teaching within the context of this paper addresses those teacher 
behaviors that facilitate student acquisition of concepts, processes, and 
skills through active involvement with general inquiry strategies. It 
incorporates aspects of the investigative and discovery phases of science 
and affords opportunities for the students to test and refine concept 
meanings.. Through this type of learning, and the acquisition and synthesis 
of scientific knowledge and processes, the ability to perform scientific 
inquiry becomes possible. A teacher equipped to engage in inquiry teaching 
would' possess questioning skills that are divergent, have a knowledge of 
science processes and have the capacity to conduct student-centered 
inductive learning activities. Specific outcome criteria which appeared in 
the research reviewed for this analysis and judged as falling within tho. 
confines of this definition include the following: knowledge of science 
processes, inquiry instructional strategy, indirect verbal behavior, 
accepting interpersonal behaviors, increased wait-time questioning 
behavior, higher cognitive level questioning behavior, and discovery 
instructional strategy. This meta-analysis was limited to studies 
involving at least one 'of these outcomes associated with inquiry strategy. 
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LITERATURE SEARCH AND SELECTION OF STUDIES 



Because this meta-analysis was focused upon inquiry teaching , research 
dated between 1965 and 1980 was considered. This period parallels the 
implementation period for the modern science curriculum projects* 

A second limitation was suggested by the work of Munby (1980). He 
concluded that "on the basis of this analysis there are grounds for viewing 
research on the affective outcomes of science education with misgiving, 
simply because there seems little to be said of the instruments' as to 
enlist our confidence in their use." Furthermore, if we are attempting to 
change attitudes with an eye toward teacher practice, a review of research 
on changing the attitude of student teachers by Morrisey (1981). is 
relevant. He claimed that the lack of change in the teaching of eJLementary 
school science indicates something more than just immediate attitude change 
must be considered. Therefore, studies involving only affective outcomes 
were not included in this meta-analysis. 

The literature search process began with disserations. Dissertations 
listed in the Science Education Dissertation Bibliography (1978) were J. 
identified. 

Journal articles were identified applying the same criteria as for 
dissertations by scanning the table of contents of the Journal of Research 
"in Science Teaching and Science Education for the years 1965 to' 1980. 
Research from sources other than dissertations and journals was identified 
through a search of the three ERIC compilations of abstracts from Resources 
in Education (1966-1972, 1973-197*5, 1976-1977) and Resources in Education 
for later years. In addition, NARST Annual Reviews of Research and NARST 
Abstracts for many years were reviewed to identify potential studies. 

A number of studies identified were rejected for the following reasons: 
(1) data needed for the calculation of effect size were incomplete, (2) 
measured outcomes 0 were only in the affective domain, (3) the native 
language of the subjects was other than English, (4) the study was produced 
prior to 1965, or (5) the .outcome variables were not associated with 
inquiry teaching strategy. 



SELECTION OF STUDIES, CODING OF VARIABLES, AND EFEECT SIZE CALCULATIONS 



Sixty-eight studies were selected for analysis. Relevant variables were 
identified and coded according to the following six major categories: 
study form and design characteristics, teacher/ teacher trainee 
characteristics, student characteristics, treatment characteristics, 
outcome characteristics, and effect size calculation characteristics. 
These categories and six variables for the management of the data deck 
resulted in the delineation of 114 variables. 
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Effect size calculations were performed using the most straight forward 
method possible w$th the data presented in each study. The most 
straight-forward method available and the one used in 64% of the effect 
size calculations involved standardization of the mean differences between 
treatment and control groups • 

ANALYSIS OF DATA 



The 68 studies were coded; these resulted in 177 effect size calculations. 
Outcomep were measured on teachers, on students, and on students about 
teachers. While many researchers advocate measuring teacher behavior by 
evaluating student performance, this practice occurred in a very small 
number of cases. One hundred and fifty-four effect sizes were outcomes 
measured on teachers, whije only nineteen were outcomes measured on 
students and four were student measures about teachers. Because of the 
small sample size, no analysis of the effect sizes related to outcomes 
measured on teachers by students was performed. 

While many science educators advocate measuring teacher performance by 
measuring student outcomes, this practice occurred infrequently in this 
collection of studies on science teacher education. The nineteen effect 
sizes related to outcomes measured on students produced a mean effect size 
of .44 and a standard deviation of .67 when broken down across all* 
variables. The mean reliability of the measurement instruments was .82 
with a standard deviation of .06. The outcome most frequently measured 
(47% of the cases) was the knowledge of science processes. 

The one hundred and fifty-four effect sizes related to outcomes on teachers 
produced a mean effect size of .85 with a standard deviation of 1.30 when 
broken down across all variables* One effect size determination was 
considered a far outlier, having a value three time greater than, the next 
closest effect size and being approximately ten standard deviations above 
the mean. If this value is discarded, the mean effect size across all 
variables becomes .77 with a standard deviation of .86. 

The size of the teacher samples ranged from 9 to 299, with a mean of 60.4 
ami a standard deviation of 45.2. The number of teachers assigned to each 
treatment ranged from 5 to 129, with a mean of 26.8 and a standard 
deviation of 17.6. Test reliability was reported for 50" percent of the * 
'outcome 'measurement -instruments, yielding a mean of .81 and a standard 
deviation of .09. Duration of treatment (with seven missing cases) ranged 
from less than a day to one year. The mean was 7 H 0.0 days with a standard 
deviation of 71.4. 

Descriptive reporting of teacher and/or student characteristics which might 
affect outcome measures was sporadic and occurred with frequencies too low 
to support analysis. Therefore the analysis was divided into the following 



three major categories: (a) variables associated with methodological 
aspects of the studies, (b) variables associated with study treatments, and 
(c) variables associated with the variety of teacher education outcomes 
sought. 



Teacher Outcomes - Methodological Variables 

Table I presents the means, standard deviations, and sample sizes of 
selected methodological variables broken down across outcomes measured on 
teachers. Table II presents the Pearson correlation coefficients and 
significance level between selected methodological variables and between 
selected methodological variables and effect size. In situations where the 
variable categories did not reflect an ordinal relationship the categories 
were coded dichotoraously for correlational purposes. 

Table I presents nine methodological variables with the categories that had 
sufficient sample size to warrant discussion. The form of publication 
variable indicates that the source of the information used for coding 
purposes. An attempt was made to locate the primary source whenever 
possible and if a study was presented as both a journal article and a 
dissertation, the dissertation was used as the source. The mean effect 
size calculated from journal articles was 1.01 while those calculated from 
dissertations was .59. There was significant positive correlation 
(p*0.0004) between journals as a source and effect size and a significant 
negative correlation (p=0.05) between dissertations as a source and effect 
size. These results indicate that a meta-analysis based solely on journals 
as a source has the^ potential of establishing effect size data higher than 
what might be expected if the extensive dissertation literature were used 
also. 

The type of study variable was 'coded using the guidelines established by 
Campbell and Stanley (1963). Pre-experimental designs consisted of 
one-group pretest-posttest designs and static-groups comparison designs. 
Experimental designs involved pretest-posttest control groups designs and 
posttest-only control group designs with random assignment of subjects to 
treatment and control groups. Quasi-experimental designs involved the same 
designs as the experimental groups, but without random assignment of 
subjects. The mean effect size ranged from .90 for pre-experimental to .67 
for experimental studies. The correlation coefficient was not significant 
at the .05 level. 

The assignment of teachers to treatments was coded as random, matched, 
self-selected, intact groups, representative sample, and other. Random 
assignment and intact group assignment together accounted for 76% of the 
effect sizes produced. Random assignment studies had an average effect 
size of .67 and intact groups produced an effect size mean of .88. 

The teacher unit of analysis variable indicated the unit (individual or 
group) that was used in establishing the degrees of freedom for the 
determination of significance level. The teacher unit of analysis used the 
number of individuals involved to determine degrees of freedom in 95% of 
the cases. 
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Table I 

Teacher Outcome Effect Sizes 
Across Study Methodological Variables 



Mean Standard 

Variable Effect Size Deviation N 

Form of publication 

Journal 1.01 .98 61 

Dissertation .59 .77 85 

Other .75 .24 7 

Type of study 

Pre-experiraental .90 .73 22 

Quasi-experimental .78 .83 69 

Experimental .67 .92 60 

Assignment of teachers to 
treatments 

Random .67 .91 61 

Self-selected .57 .78 17 

Intact groups * .88 «86 56 

Teacher unit of analysis 

Individual .77 .88 145 

Classroom or group .75 1.00 4 

Rated internal validity 

Low .55 .55 25 

Medium .75 .82 55 

High .82 .98 68 

Design Rating 

Low .57 .80 41 

Medium .68 .61 38 

High .92 .99 72 

Outcome instrument type 

Published, national stand. .35 .58 16 

Ad hoc, for that study 1.12 .94 59 

Other .60 .76 77 

Measurement method 

Multiple choice .48 .61 31 

Likert .50 .44 24 

Observation .84 .93 49 

Other 1.14 1.07 35 

Time of measurement 

After treatment ;62 .76 60 

Pre-post .90 .96 80 

Other .74 .59 9 
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Table II 

Correlations Among Selected Methodological 
Variables and Between Methodological Variables and Effect Size 



Variable A 


Variable B 


r 


P 


N 


Form of pub. journal 


effect 


size 


0.23 


0.004 


61 


Form of pub. journal 


extent 


of treat 


0.21 


0.008 


61 


Form of pub. journal 


it of teachers 










assigned 


A A / 

-0 .04 


G.6J4 


01 


Form of pub. journal 


it of teachers 










analyzed 


a aa 

-0 .00 


A Q CO 

o.y 59 


01 


Form of pub. journal 


reported sig. 


-0.22 


0.010 


61 


Type of study 


effect 


size 


-0.09 


0.248 


151 


Rated internal validity 


effect 


size 


0. 10 


0.214 


I/O 

148 


Design rating 


effect 


size 


0.17 


0.035 


151 


• 

Outcome instrument, pub* 












national standardized 


effect 


size 


-0.17 


0.039 


16 


Outcome instrument, ad hoc 


effect 


size 


0.32 


0.000 


59 


Outcome instru. other 


effect 


size 


-0.20 


0.012 


77 


Measurement method 












multiple choice 


ef feet 


size 


-0.17 


0.039 


31 


Measurement method 












other 


ef feet 


size 


0.24 


0.003 


35 


it of teachers assigned 


effect 


size 


-0.21 


0.011 


152 


it of teachers analyzed 


effect 


size 


-0.17 


0.031 


153 


# of teachers on outcome 












measure 


effect 


size 


-0.17 


0.035 


149 
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An internal validity variable addressed the assignment of individuals to 
treatments, and the percent mortality among treatment and control groups. 
Studies were rated high in internal validity if group equivalence was 
established through random assignment or other procedures and subject 
mortality was less than 15 percent. Studies were coded as medium if (a) 
randomization was not used but low mortality was maintained, (b) 
randomization existed but mortality was high or nonequivalent , and (c) if 
random procedures were used in the selection of intact groups and mortality 
was lov. Studies were coded as low where intact convenience samples were 
used and/or where the existence of factors confounding the equivalence of 
the subject groups was apparent. Studies rated low in validity produced 
effect sizes with a mean of .55, medium validity studies produced effect 
sizes with a mean of ,75, and high validity studies produced effect sizes 
with a mean of .82. The internal validity was positively correlated with 
effect size, but was not significant at the .05 level. 

The design rating variable addressed the apparent degree of control of the 
confounding variables by the experimental procedure. Studies were rated as 
high if the design established control to the extent that post-treatment 
differences could be attributed to treatment effects. The study was rated 
as medium if the design indicated lack of control of a variable that 
probably contributed to some outcome differences. The study was rated as 
low if the failure to control a given variable obviously contributed to 
outcome differences. Studies with a low- design rating had an average 
effect size of .80, medium, studies had an average effect size of .61, and 
high studies had an average effect size of .99. A positive correlation 
coefficient with a significance level of .035 indicated that larger effect 
sizes were associated with higher design ratings. 

A variable, outcome instrument type, included instruments categorized as 
(a) published, (b) ad hoc developed for that particular study, or (c) 
other. Most of the tests in the "other" category were developed ad hoc for 
another study and then used in existing form or with modifications for the 
study being coded. The effect sizes that resulted from ad hoc instruments 
produced a mean effect size of 1.12. Other instruments produced an effect 
size of .60 and published instruments a mean of .35. The ad hoc 
instruments had a significant positive correlation with effect size (p > 
0.001) and the published and other categories had a significant negative 
correlation with effect size (p > .05). While these notable higher effect 
sizes for the specially designed instruments corld be due to investigator 
bias, it seems more likely the result of the instruments being better 
designed to detect outcome differences to which the given study is 
directed. 

Measurement method categories produced the following results for mean 
effect size: multiple choice .48, Likert .50, observation .84, and other 
1.14. Multiple choice methods correlated negatively with effect size with 
a significance of .039. The "other" category correlated positively with 
effect size with a significance of .003. The remaining categories did not 
correlate significantly at the .05 level. 
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The time of measurement variable included the following categories: (a) 
after treatment, (b) pre-post, and (c) other. The latter category was used 
'when different instruments on the same outcome were averaged to determine 
one effect size. The pre-post measurement produced the largest mean effect 
size, .90. The "other" category had a mean effect size of .74 and the 
after treatment category .62. # 

The number of teachers assigned to the study, the total number analyzed, 
and the number measured on each outcome instrument correlate negatively 
with effect size (significant at level o^f p > .05). The journal category 
of the form of publication varic K le did/not correlate significantly at the 
.05 level with any of the variabxes related to sample size, but the journal 
category did correlate significantly with the reported significance 
variable and the extant of treatment 'variable. The extent of treatment 
variable addressed the scope of the t T aatment with a multi-grade treatment, 
a program, or an on-going institute' .being at the broad end of the scale and 
a specific training technique being at the narrow end of the scale. These 
correlations indicated * that studies taken from journals had low values for 
p and that these studies addressed treatment types of uarrow scope. 

Table III presents the average effect size, standard deviations and 
correlation coefficients 'for two of the variables related to effect size 
calculations: (a) source of means i.e., unadjusted posttest, pre-post 
difference or other and (b) method for calculation. Means for the 
calculation of effect size were unadjusted post-test means in 83 cases and 
provided a mean effect size of .62 with a standard deviation of .74. Means 
were a result of pre-post differences in 47 cases with a raeajl effect size 
of 1.00 and a standard deviation of 1.09. The "other" category involved 
pre- experimental studies wherein pre-test data were used to generate a 
control group mean. This category provided a mean effect size of .88 and a 
standard deviation of .75. The unadjusted post-test source of means had a 
negative correlation with effect size significant at the .05 level. The 
source of means in the pre-post category had a positive correlation with 
effect size significant at the .05 level also. 

The methods used for calculating the effect size included the following: 
(a) calculating directly from reported means and variances or from raw 
data, (b) calculating with direct estimates of the variance from ANOVA, t, 
and F values, (c) calculating using reported probability levels, and (d) 
calculating using pre-test data as a control group. Using pre-test data as 
a control group produced the highest mean effect size, 1.01 with a standard 
deviation of .78. The means and standard deviations for using directly 
reported means and variances and direct estimates were close, having mean 
effect values of .72 and .84 respectively. None of the calculation method 
variables showed a significant correlation with effect size at the .05 
level. 



Table III 



Teacher Outcome Effect Sizes 
Across Effect Size Calculation Variables 



Variable 

Source of means unadjusted 

post-test 

Source* of means pre-post 

differences 

Source of means other 

Calculated directly from reported 

values or raw data 
Calculated with direct estimates 

(AN OVA, t, F) 
Calcuated frgm reported 

probability levels 
Calculated using pre- test data as 

a control group 



Mean Standard 

Effect Size Deviation N 

.62 .74 83 

1.00 1.09 47 

.88 .75 19 

.72 .90 96 

.84 .87 34 

.71 .15 5 

1.02 .78 15 



Correlation with Effect Size 



Variable 

Source of means unadjusted 

post-test 
Source of means pre-post dif. 

Calculated directly from reported 

values or raw data 
Calculated with direct estimates 

• (ANOVA, t, F) 
Calculated from reported 

probability levels 
Calculated using pre-test data 

as a control group 



r p N 

-0.1865 0.021 83 

0.1745 0.031 19 

-0.0694 0.394 96 

0.0420 0.606 34 

-0.0131 , 0-872 • 5 

0.0948 0.244 15 
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Summary 



Many methodological variables were correlated with effect sizes. Nearly 
all the results of the correlations agree with the general findings of the 
other meta analysis studies when these variables were analyzed (Anderson, 
1982). Methodological variables t correlating positively with effect size at 
the .05 level (or higher) of significance were (1) studies published in 
journals, (2) better research designs, (3) the use of specially designed 
instruments (rather than standardized, nationally used instruments), and 
(4) studies involving smaller numbers of teachers. High internal validity 
correlated positively but was not significant at the .05 level. 

These results indicate that the design of the study, the sample size (or 
the intensity of the 'instruction) , and the appropriateness of the 
instrument for the outcomes being measured all relate to the level of the 
effect sizes in a significant way. The relationships found in this 
meta-analysis and in other meta-analysis studiea provide guidance for 
designing effective instruction. Some cautions are suggested in 
interpreting non-significant results in research that have not considered 
these relationships; the non-significant finding may not be due to the 
instruction, but rather to the problem of sample size (or intensity of 
instruction) and inappropriate instruments. Careful analysis of 
non-significant studies may provide clues for studies that might give 
significant results with better instrumentation and more attention to 
sample size and intensity of instruction. 

The correlations .of sample size to effect size help- to explain why some 
effective research results are hard to implement effectively with larger 
samples of teachers. Providing adequate instruction, getting teachers to 
follow guidelines, and providing feedback for a large sample is much more 
difficult than working with a small sample? 

The significant correlation of journal articles with effect size presents a 
caution to people doing secondary analysis of research. The primary 
literature usually contains more non-significant studies and studies with 
smaller effect sizes. While these additional studies from the primary 
literature may not change the conclusions in all meta-analysis studies, the 
differences encountered in some studies such as this one indicate that 
there is a good possibility that different conclusions might result without 
the use of primary' literature data. 



Teachjer Outcomes - Treatment Variables 

Tables IV and V present the effect sizes of teacher outcomes broken down 
across various treatment variables. Table VI presents correlations between 
treatment variables and effect size. The first of these variables, time- of 
treatment, was categorized as pre-service and in-servfce. These two groups 
produced mean effect sizes of .78 and .72 respectively. 



Table IV 

Teacher Outcome Effect Sizes 
Across Treatment Variables (Part I) 



Variable 

Time of treatment 
Pre-service 
In-service 

Site of treatment 

Field-based, site of 
employment 

Field-based, not the 
site of employment 
University-based * 



Mean 
Effect Size 



.78 
.72 



.74 

.77 
.77 



Standard 
Deviation 



.90 
.74 



.86 

.60 
.88 



N 



122 
31 



20 
112 



Extent of treatment 

Multi-grade or level, 
e.g. , program or ongoing 
institute 

One grade level, e.g., 
course or workshop 
Training technique 



.45 

.75 
.84 



.45 

.78 
.98 



12 

69 
72 



Treatment geared to grade level 
Elementary school 
Secondary 
General 

Number of variables used to 
describe eath treatment 

1 

2 

3 

4 



.76 
.39 
1.24 



.67 
.65 
.73 
1.25 



.86 
.32 
.97 



.57 
.89 
.62 
1.03 



123 
8 

•15 



42 
64 
31 
14 
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The site of treatment variable categorized field-based treatments versus * 
university based treatments* The field-based treatments category was 
further divided into treatments that ocurred in the schools in which the 
teachers were employed versus treatments in schools where the individuals 
were not employed. Treatments were predominately university-based, 
accounting for 77% of the reported effect sizes. The mean effect sizes, for 
the three groups were very close in value ranging from .74 to .77. 

The extent of treatment variable ranged from a broad scope addressing 
multi-grade or program treatments to treatments that focused on a 
particular training technique. The multi-grade or program level produced a 
mean effect size of .45, the one grade or level variable produced a mean 
effect size of .75, and the training technique level produced a mean effect 
size of .84. 

* 

The "treatment geared to grade level" variable categorized the target 
population where the treatment outcomes were to be applied. % The elementary 
level group accounted for 81% of the effect sizes coded and produced a mean 
effect size of .76 with a standard deviation of .86. Those treatments 
categorized as secondary had a mean effect size of .39 and those 
.categorized as general had a vaj.ue of 1.24. 

In many instances more than one treatment variable was used to classify a 
treatment. Those treatments that were described using one variable 
produced a mean effect size of .67; two variables, .89; three variables, 
.73; and four variables, 1.25. A positive correlation between this 
variable and effect size occurred at a probability level p > .001. 

Of the many treatment variables, those designated here as "treatment type" 
are of particular interest. The data on these variables is found in Table 
V; this Table, in contrast to the previous Tables, includes all variables 
regardless of the number of effect sizes recorded. 

Treatment type variables were divided into the following sections: 
organizational pattern, type of instruction, mode of instruction, source of 
structure, locus of control, training techniques and technology employed. 
The first of these categories refers to the form of organizational pattern 
within which the instruction was offered. The next five all refer to the 
type of instruction offered. 

Some treatments were categorized in terms of the treatment organizational 
pattern and included the following: field-based programs, workshops, 
methods courses, science courses designed for teachers, and units of study. 
Those variables with an N of more than three included the field-based 
program category with a mean effect size of .35, the workshop level at .73, 
methods courses at .79, science courses designed for teachers at .97, and 
specific units of study at 1.38. The type of instruction category pertains 
to the instructional approach used fn the teacher education activities. If 
the treatment involved instruction versus no instruction with no further 
delineation of approach, it was categorized as general. Other approaches 
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Table V 

Teacher Outcome Effect Sizes Across Treatment Variable (Part II) 



Treatment Type 






8 


n 


Organizational Pattern 


Field-based Program 


.35 


.40 


8 




Ongoing Institute 


.64 


.48 


2 




Summer Institute 


.14 


.09 


3 


• 


Workshop 


.73 


.75 


16 




Methods Course 


.79 


.94 


22 




Science Course 


< 1.28 


.48 


2 




Science Course Designed 










for Teachers 


.97 


.•70 


9 




Units of Study 


1.38 


1.29 


22 


Type of Instruction 


General 


.79 


1.21 


35 




Traditional 


.30 


.32 


5 


- 


Inquiry 


.63 


.63 


9 




Discovery 


.40 


.29 


7 


Mode of Instruction 


Verbal 


-.03 


.18 


2 




Mixed 


.45 


.86 


• 12 




Concrete 


.75 


.75 


20 


Source of Structure 


Student Self-Directed 


.04 


.46 


8 




Student; Interacting with 










Teacher and /or Materials 


.70 


1.01 


8 




Teacher 






0 




Criterion ref e renced 


.69 


.02 


2 


Locus of Control 


Student self-directed 


.82 


.88 


4'4 


c 


Teacher directed 


1.44 


0 


1 




Mix, part student, part 










teacher 






0 


Training Technique 


Interaction Analysis 










Feedback 


1.33 


0 


1 




Instructional Strategy 










Feedback 


.67 


.91 


10 




Wait-Time Analysis 


3.95 


.07 


2 




Questioning Analysis 


1.38 


1.65 


8 




Micro-teaching Peers 


.72 


.35 


4 




Micro-teaching Students 


.81 


' .52 


6 




Modeling Strategy 


1.56 


1.19 


14 




Behavior Coding Training 










(e.g. IA) or Strategy 




* 






Analysis 


1.37 


. .87 


8 


Technology Employed 


Audio Technology 


1.04 


.25 


4 




Video Technology 


1.82 


1.44 


'9 




Programmed Material 










(Audio-Tutorial) 


.99' 


.76 


17 




Print Material 


1.40 


0 


1 
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Table VI 



Correlations between Selected Treatment Variables 
and Effect Size 



Variable r p~ 

# of variables describing treatment 0.3123 0.000 
treatment units of study . * 0.2884 0.000 

source of structure student 

* self-directed 1 -0.2003 0.013 

questioning analysis ^0.1664 0.040 

modeling strategy - 0.2928 0.000 < 

behavior coding training or 

strategy analysis 0.1637 0.043 

video technology * x - 0.3039 0.000 

outcome science processes 0.1871 0.021 

outcome questions process directed 0.1962 0.015 
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Table VII 

Teacher Outcome Effect Sizes by Type of Outcome 

Type of Outcome m s n 

Knowledge and Intellectual 



Processes < Science Content 


.52 


.79 


7 


Science Processes 


1.08 


1.03 


33 


Methods of Science and the 








Scientific Enterprise 


.14 


.74 


3 


Critical Thinking 


.09 


0 


1 


Creativity 


.19 


0 


i 


Problem Solving 


.04 


.23 


5 


Behavioral Objectives 


.75 ■ 


.14 


3 


Planning (organizational 








skill) 


.90 


.12 


2 


* 

Composite Knowledge and 


j 






Intellect 


J .80 




55 


Teacher Classroom Behavior Verbal Behavior, General 


.15 


0 


1 


Inquiry Strategy 


.89 


'.'47 


4 


Concrete Manipulative 








Strategy . 


1.26 


0 


1 


Indirect VerbaJL Behavior 


.72 


.82 


18 


Interpersonal Behaviors 


.54 


.26 


5 


Quest ion ing-level 


.72 


1.18 


13 


„ * Discovery Strategy (Student 








Centered, Open) 


.70 


.53 


7 


Group Process Skills 


.26 


0 


1 


Question - Process Directed 


1.45 


.60 


9 


Reactions to Classroom 








Situations 


.84 


0 


1 


Composite Teacher Classroom 








Behaviors 


.82 




60 


Affective Attitude (general) 


.79 


.56 


6 


Attitude toward Science 


.39 


.29 


10 


Attitude toward Science 








Teaching 


.09 


.21 


4 


Attitude toward Treatment 


.46 


0 


1 


Dogmatism (toward open) 


.34 


.34 


5 


Philosophy of Teaching 








(toward student centered) 


.72 


.65 


4 


Attitude toward Treatment 








" Emphasis 


.60 


0 


1 


Composite Affective 


.47 
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were termed traditional, inquiry, and discovery. The classification as 
inquiry or discovery was made ,from 'the" language used in the study coded 
even though the terms were considered as synonymous in jthis report ♦ The 
general instructional category produced a mean effect size of .79, 
traditional instruction had an, effect size of .30, the inquiry category had 
a mean effect size of .63, and the^ discovery approach had an N of #40. 

The mode of instruction categorized the approaches as predominately verbal, 
predominately concrete, indicating a high level of student involvement with^ 
manipulative exercises; or mixed, involving both the verbal and the 
concrete. Little data were found in the verbal category but the concrete 
vcode produced a mean effect size of .75 and the mixed mode produced a mean 
effect size of .44. 

The source of structure concept addressed the source of instructional 
objectives, content, and/or method used in the treatment. The categories 
included student self-directed, student interacting with materials /and or 
the teacher, the teacher as the source of structure, and criterion 
referenced sources. The source of structure involving student 
self-direction produced a very low average effect size, .04. The structure 
that involved interaction of the student with materials and/or the teacher 
produced a mean effect size of .70. A similar effect size (.69) was found 
for "criterion referenced," although this result is based on only two 
studies. 

The locus of control concept addressed the approach used in meeting the 
objectives, etc., set forth in the structure component. The categories 
included student self-directed,- student and teacher working together^ 
teacher directed, and a mix of part student and part .teacher directed. The 
student self-directed category produced a mean effect size of '.81, based on 
44 effect sizes. Though a higher value is reported for "teacher directed," 
it is based on only one study. 

-The training technique concept addressed educational practices usually 
employed within the confines of a course or workshop. This concept 
included the following categories: interaction analysis feedback, 
instructional strategy feedback, wait-time analysis, questioning analysis, 
micro-teaching* peers,' micro-teaching students, modeling strategy, and 
behavior coding training or strategy analysis. Instructional strategy 
feedback produced a mean effect size of .67; modeling strategy, 1.56; 
micro- teaching peers, .72; micro-teaching students, .81; behavior coding 
training analysis, 1.37; and questioning analysis, 1.38. 

The technology concept addressed the use of audio technology, video 
technology, programmed material or auto-tutorial methods as treatment 
variables. Treatments using audio technology had a mean effect size of 
.99. 
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Suamary 

The data analyzed in this study tend to agree with other meta-analysis 
studies (Anderson, 1982) and other reviews of research (Helgesoh, Blosser, 
and Howe, 1977)* Higher effect sizes were generally obtained wit;h (1) more 
specific instruction, (2) instruction that involved a substantial amount of 
concrete experience , and (3) instruction involving elementary school 
teachers* 

Positive gains were made by groups in both preservice and inservice 
instruction, and using a variety of techniques. 

These results suggest that some alternatives to more traditional 
instruction should be explored. They may produce equal or better gains and 
save resources. Studies that involved analysis of instruction (modeling, 
behavior -analysis, and questioning analysis) tended to have the highest 
effect sizes. Work with elementary teachers showed substantial gains, 
suggesting that they probably had few inquiry skills and gained. from the 
instruction provided. 



Teacher Outcomes - by Outcome Category 

Outcome criteria were classified into the following three categories: 
criteria related to knowledge and intellectual processes, criteria related 
to classroom teacher behaviors, and criteria related to affective outcomes. 
Information on teacher outcome effect sizes by type of outcome is found in 
rable VII. 

In the knowledge and intellectual processes category, knowledge of science 
processes was by far the most commonly measured. It was measured in 33 
cases and produced a mean effect lize of 1.08. Other outcome variables 
were measured much less frequently, as shown in Table VII. The knowledge 
and intellectual processes category, overall, produced an average effect 
size of .80 based on 55 effect sizes. 

The measurement of outcome variables in the teacher classroom behavior 
category was more varied with six different variables being measured in 
four or more studies. The variable inquiry strategy had a mean effect size 
of .89; indirect verbal behavior, .72; interpersonal behaviors, .54; 
questioning-level, .72; discovery* strategy, .70; and questions (process 
directed), 1.45. The teacher classroom behavior category* overall produced 
a mean effect size of .82 based on 60 effect, sizes. 

A variety of affective measures was used in these teacher education studies 
including measures of attitudes toward science, science teaching, and 
several others. The average effect size in these categories varied from 
.09 to #79, with a mean overall effect size for the affective category of 
.47 based on 31 effect sizes. 
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Summary 



Teaching science processes appeared to be a realistic goal, effectively 
taught in many studies* The mean of effect size, 1.08, was the highest 
mean in the knowledge and intellectual processes category. While some 
other variables had mean effect sizes from .5 to .90, a few studies 
(particularly problem solving) had low effect sizes. 

Teacher classroom behavior appeared to be effective in many studies. 
Nearly all the skills and processes that were taught had mean effect sizes 
of .5 to over 1.0. Questioning skills and other verbal skills had mean 
effect sizes from .7 to 1.45. 

Affective variables appeared to be more difficult to teach or to gain large 
effect sizes. Most means ranged from about .34 to .7. 

These results are very similar to the conclusions of the review of research 
by Balzer, Evans, and Blosser (1973). Cognitive (low level) 'and skill 
outcomes are usually easiest to teach. Complex behaviors and affective 
outcomes are usually hatder to teach. The data indicate that many 
treatments were able to achieve respectable effect sizes, emphasizing a 
variety of learning outcomes. Instruction with groups involving fewer 
students were usually more effective than for groups with larger numbers of 
students. 
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